With the swift advancement of neural networks and their expanding applications in many fields, optical neural networks have gradually become a feasible alternative to electrical neural networks due to their parallelism, high speed, low latency, and power consumption. Nonetheless, optical nonlinearity is hard to realize in free-space optics, which restricts the potential of the architecture. To harness the benefits of optical parallelism while ensuring compatibility with natural light scenes, it becomes essential to implement two-dimensional spatial nonlinearity within an incoherent light environment. Here, we demonstrate a lensless opto-electrical neural network that incorporates optical nonlinearity, capable of performing convolution calculations and achieving nonlinear activation via a quantum dot film, all without an external power supply. Through simulation and experiments, the proposed nonlinear system can enhance the accuracy of image classification tasks, yielding a maximum improvement of 5.88% over linear models. The scheme shows a facile implementation of passive incoherent two-dimensional nonlinearities, paving the way for the applications of multilayer incoherent optical neural networks in the future.

On-chip diffractive optical neural networks (DONNs) bring the advantages of parallel processing and low energy consumption. However, an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method. Moreover, the loss caused by the open boundaries poses challenges to applications. A multimode DONN architecture based on a more precise eigenmode analysis method is proposed. We have constructed a universal library of input, output, and metaline structures utilizing this method, and realized a multimode DONN composed of the structures from the library. On the designed multimode DONNs with only one layer of the metaline, the classification task of an Iris plants dataset is verified with an accuracy of 90% on the blind test dataset, and the performance of the one-bit binary adder task is also validated. Compared to the previous architectures, the multimode DONN exhibits a more compact design and higher energy efficiency.

Ever-growing deep-learning technologies are making revolutionary changes for modern life. However, conventional computing architectures are designed to process sequential and digital programs but are burdened with performing massive parallel and adaptive deep-learning applications. Photonic integrated circuits provide an efficient approach to mitigate bandwidth limitations and the power-wall brought on by its electronic counterparts, showing great potential in ultrafast and energy-free high-performance computation. Here, we propose an optical computing architecture enabled by on-chip diffraction to implement convolutional acceleration, termed “optical convolution unit” (OCU). We demonstrate that any real-valued convolution kernels can be exploited by the OCU with a prominent computational throughput boosting via the concept of structral reparameterization. With the OCU as the fundamental unit, we build an optical convolutional neural network (oCNN) to implement two popular deep learning tasks: classification and regression. For classification, Fashion Modified National Institute of Standards and Technology (Fashion-MNIST) and Canadian Institute for Advanced Research (CIFAR-4) data sets are tested with accuracies of 91.63% and 86.25%, respectively. For regression, we build an optical denoising convolutional neural network to handle Gaussian noise in gray-scale images with noise level σ=10, 15, and 20, resulting in clean images with an average peak signal-to-noise ratio (PSNR) of 31.70, 29.39, and 27.72 dB, respectively. The proposed OCU presents remarkable performance of low energy consumption and high information density due to its fully passive nature and compact footprint, providing a parallel while lightweight solution for future compute-in-memory architecture to handle high dimensional tensors in deep learning.

Non-line-of-sight (NLOS) imaging is an emerging technique for detecting objects behind obstacles or around corners. Recent studies on passive NLOS mainly focus on steady-state measurement and reconstruction methods, which show limitations in recognition of moving targets. To the best of our knowledge, we propose a novel event-based passive NLOS imaging method. We acquire asynchronous event-based data of the diffusion spot on the relay surface, which contains detailed dynamic information of the NLOS target, and efficiently ease the degradation caused by target movement. In addition, we demonstrate the event-based cues based on the derivation of an event-NLOS forward model. Furthermore, we propose the first event-based NLOS imaging data set, EM-NLOS, and the movement feature is extracted by time-surface representation. We compare the reconstructions through event-based data with frame-based data. The event-based method performs well on peak signal-to-noise ratio and learned perceptual image patch similarity, which is 20% and 10% better than the frame-based method.

In this paper, an artificial-intelligence-based fiber communication receiver model is put forward. With the multi-head attention mechanism it contains, this model can extract crucial patterns and map the transmitted signals into the bit stream. Once appropriately trained, it can obtain the ability to restore the information from the signals whose transmission distances range from 0 to 100 km, signal-to-noise ratios range from 0 to 20 dB, modulation formats range from OOK to PAM4, and symbol rates range from 10 to 40 GBaud. The validity of the model is numerically demonstrated via MATLAB and Pytorch scenarios and compared with traditional communication receivers.

Microwave photonic receivers are a promising candidate in breaking the bandwidth limitation of traditional radio-frequency (RF) receivers. To further balance the performance superiority with the requirements regarding size, weight, and power consumption (SWaP), the implementation of integrated microwave photonic microsystems has been considered an upgrade path. However, up to now, to the best of our knowledge, chip-scale fully integrated microwave photonic receivers have not been reported due to the limitation of material platforms. In this paper, we report a fully integrated hybrid microwave photonic receiver (FIH-MWPR) obtained by comprising the indium phosphide (InP) laser chip and the monolithic silicon-on-insulator (SOI) photonic circuit into the same substrate based on the low-coupling-loss micro-optics method. Benefiting from the integration of all optoelectronic components, the packaged FIH-MWPR exhibits a compact volume of 6cm3 and low power consumption of 1.2 W. The FIH-MWPR supports a wide operation bandwidth from 2 to 18 GHz. Furthermore, its RF-link performance to down-convert the RF signals to the intermediate frequency is experimentally characterized by measuring the link gain, the noise figure, and the spurious-free dynamic range metrics across the whole operation frequency band. Moreover, we have utilized it as a de-chirp receiver to process the broadband linear frequency-modulated (LFM) radar echo signals at different frequency bands (S-, C-, X-, and Ku-bands) and successfully demonstrated its high-resolution-ranging capability. To the best of our knowledge, this is the first realization of a chip-scale broadband fully integrated microwave photonic receiver, which is expected to be an important step in demonstrating the feasibility of all-integrated microwave photonic microsystems oriented to miniaturized application scenarios.